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Error Detecting and Error Correcting Codes 

By K. W. HAMMING 

1.. Introduction 

THE author was led to the study given in this paper from a considera- 
tion of large scale computing machines in which a large number of 
operations must be performed without a single error in the end result. This 
problem of "doing things right" on a large scale is not essentially new; in a 
telephone central office, for example, a very large number of operations are 
performed while the errors leading to wrong numbers are kept well under 
control/ though they have not been completely eliminated. This has been 
achieved, in part, through the use of self-checking circuits. The occasional 
failure that escapes routine checking is still detected by the customer and 
will, if it persists, result in customer complaint, while if it is transient it will 
produce only occasional wrong numbers. At the same time the rest of the 
central office functions satisfactorily. In a digital computer, on the other 
hand, a single failure usually means the complete failure, in the sense that 
M H m de t ec te d no more computing can be done until the failure is located 
and corrected, while if it escapes detection then it invalidates all subsequent 
operations of the machine. Put in other words, in a telephone central office 
there are a number of parallel paths which are more or less independent of 
each other; in a digital machine there is usually a single long path which 
passes through the same piece of equipment many, many times before the 
answer is obtained. 

In transmitting information from one place to another digital machines 
use codes which are simply sets of symbols to which meanings or values are 
attached. Examples of codes which were designed to detect isolated errors 
an numerous; among them are the highly developed 2 out of 5 codes used 
extensively in common control switching systems and in the Bell Relay 
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Computers, 1 the 3 out of 7 code used for radio telegraphy, 1 and the word 
count sent at the end of telegrams. 

In some situations self checking is not enough. For example, in the Model 
5 Relay Computers built by Bell Telephone Laboratories for the Aberdeen 
Proving Grounds, 1 observations in the early period indicated about two 
or three relay failures per day in the 8900 relays of the two computers, repre- 
senting about one failure per two to three million relay operaRons. The self- 
checking feature meant that these failures did not introduce undetected 
errors. Since the machines were run on an unattended basis over nights and 
week-ends, however, the errors meant that frequently the computations 
came to a halt although often the machines took up new problems. The 
present trend is toward electronic speeds in digital computers where the 
basic elements are somewhat more reliable per operation than relays. How- 
ever, the incidence of isolated failures, even when detected, may seriously 
interfere with the normal use of such machines. Thus it appears desirable 
to examine the next step beyond error detection, namely error correction. 

We shall assume that the transmitting equipment handles information 
in the binary form of a sequence of 0's and l's. This assumption is made 
both for mathematical convenience and because the binary system is the 
natural form for representing the open and closed relays, flip-flop circuits,| 
dots and dashes, and perforated tapes that are used in many forms of com- 
munication. Thus each code symbol will be represented by a sequence of 
0's and l's. 

The codes used in this paper are called systematic codes. Systematic codes 
may be defined 1 as codes in which each code symbol has exactly n binary 
digits, where m digits are associated with the information while the other 
k =» n — m digits are used for error detection and correction. This produces 
a redundancy R denned as the ratio of the number of binary digits used to 
the minimum number necessary to convey the same information, that is, 

R = n/nt. 

This serves to measure the efficiency of the code as far as the transmission 
of information is concerned, and is the only aspect of the problem discussed 
in any detail here. The redundancy may be said to lower the effective channel 
capacity for sending information. 

The need for error correction having assumed importance only recently, 
very little is known about the economics of the matter. It is clear that in 

1 Franz Alt, "A Bell Telephone Laboratories' Computing Machine" — I, II. Mathe- 
matical Tables and Other Aids to Computation, Vol. 3, pp. 1-13 and 60-84, Jan. and 
Apr. 1948. 

1 S. Sparks, and R. G. Kreer, "Tape Relay System for Radio Telegraph Operation," 
R.C.A. Review, Vol. 8, pp. 393-426, (especially p. 417), 1947. 

1 In Section 7 this is shown to be equivalent to a much weaker appearing definition. 
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using such codes there will be extra equipment few encoding and correcting 
errors as well as the lowered effective channel capacity referred to above. 
Because of these considerations applications of these codes may be expected 
to occur first only under extreme conditions. Some typical situations seem 
to be: 

a. unattended operation over long periods of time with the minimum of 
standby equipment. 

b. extremely large and tightly interrelated systems where a single failure 
incapacitates the entire installation. 

c. signaling in the presence of noise where it is either impossible or un- 
economical to reduce the effect of the noise on the signal. 

These situations are occurring more and more often. The first two are par- 
ticularly true of large scale digital computing machines, while the third 
occurs, among other places, in "jamming" situations. *** 

The principles for designing error detecting and correcting codes in the 
cases most likely to be applied first are given in this paper. Circuits for 
implementing these principles may be designed by the application of well- 
known techniques, but the problem is not discussed here. Part I of the paper 
shows how to construct special minimum redundancy codes in the follow- " 
ing cases: 

a. single error detecting codes 

b. single error correcting codes 

c. single error correcting plus double error detecting codes. 

Fart II discusses the general theory of such codes and proves that undo' 
the assumptions made the codes of Part I are the "best" possible. 

PART I 
SPECIAL CODES 

2. Single Eeros. Detecting Codes 

We may construct a single error detecting code having n binary digits 
in the following manner: In the first n — 1 positions we put n — 1 digits of 
information. In the *t-th position we place either or 1, so that the entire n 
positions have an even number of l's. This is clearly a single error detecting 
code since any single error in transmission would leave an odd number of 
l's in a code symbol. 

The redundancy of these codes is, since m «* » — 1, 
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It might appear that to gain a low redundancy we should let n become veiy 
.large. However, by increasing u, the probability of at least one error in a 
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symbol increases; and the risk of a double error, which would pass unde- 
tected, also increases. For example, if p <K 1 is the probability of any error, 
then for n so large as 1/p, the probability of a correct symbol is approxi- 
mately l/e = 0.3679 . . . , while a double error has probability 1/2* = 
0.1839. . . . 

The type of check used above to determine whether or not the symbol 
has any single error will be used throughout the paper and will be called 
a parity check. The above was an even parity check; had we used an odd 
number of Vs to determine the setting of the check position it would have 
been an odd parity check. Furthermore, a parity check need not always 
involve all the positions of the symbol but may be a check over selected posi- 
tions only. 

3. Single Error Correcting Codes 

To construct a single error correcting code we first assign m of the n avail- 
able positions as information positions. We shall regard the tn as fixed, but 
the specific positions are left to a later determination. We next assign the k 
remaining positions as check positions. The values in these k positions are 
to be determined in the encoding process by even parity checks over se let ted 
information positions. 

Let us imagine for the moment that we have received a code symbol, with 
or without an error. Let us apply the k parity checks, in order, and for eat h 
time the parity check assigns the value observed in its check position we 
write a 0, while for each time the assigned and observed values disagree 
we write a 1 . When written from right to left in a line this sequence of k 0's 
and l's (to be distinguished from the values assigned by the parity rherkv 
may be regarded as a binary number and will be called the rhftkmg m*mkrf 
We shall require that this checking number give the position of any single 
error, with the zero value meaning no error in the symbol. Thus the check 
number must describe m + k + 1 different things, so that 

2* > m + k + 1 

is a condition on k. Writing n — m + k we find 

n + 1 

Using this inequality we may calculate Table I, which gives the maximum 
m for a given n, or, what is the same thing, the minimum n for a given m. 

We now determine the positions over which each of the various parity 
checks is to be applied. The checking number is obtained digit by digit, 
from right to left, by applying the parity checks in order and writing down- 
the corresponding or 1 as the case may be. Since the checking number is 
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Table I 



n 


m 


Corresponding k 


1 





1 


2 





2 


3 


1 


2 


4 


1 


3 


5 


2 


3 


6 


3 


3 


7 


4 


3 


8 


4 


4 


9 


5 


4 


10 


6 


4 


11 


7 


4 


12 


8 


4 


13 


9 


4 


14 


10 


4 


15 


11 


4 


16 


11 


5 



Etc. 



to give the position of any error in a code symbol, any position which has 
a 1 on the right of its binary representation must cause the first check to 
fail. Examining the binary form of the various integers we find 

1 = 1 
3 = 11 

5 = 101 
7 - 111 
9 = 1001 

Etc. 

have a 1 on the extreme right. Thus the first parity check must use positions 

1, 3, 5, 7, 9, • • • . 

In an exactly similar fashion we find that the second parity check must 
use those positions which have l's for the second digit from the right of their 
binary representation, 

2 = 10 
3= 11 

6 - 110 

7 = 111 

10 = 1010 

11 = 1011 
Etc., 
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the third parity check 



4 


= 


100 


5 


=± 


101 


6 


„ 


110 


7 


= 


111 


12 


= 


1100 


13 


= 


1101 


14 


= 


1110 


15 


s=s 


1111 


20 


= 


10100 




Etc. 



It remains to decide for each parity check which positions are to contain 
information and which the check. The choice of the positions 1, 2, 4, 8, • * • 
for check positions, as given in the following table, has the advantage of 
making the setting of the check positions independent of each other. All 
other positions are information positions. Thus we obtain Table II. 

Table II 



Check Number 


Check Positions 


1 


1 


2 


2 


3 


4 


- 4 


8 - 



Positions Checked 



1,3, 5, 7,9, 11, 13, 15, 17,-- 
2,3,6,7, 10, 11, 14, 15, 18, ■ 
4, 5,6,7, 12, 13, 14, 15, 20, • . 
8,9, 10, 11, 12, 13, 14, 15, 24, 



As an illustration of the above theoYy we apply it to the case of a se\«*n 
position code. From Table I we find for n — 7, m — 4 and k = 3. From 
Table II we find that the first parity check involves positions 1, 3, 5, 7 and 
is used to determine the value in the first position; the second parity check, 
positions 2, 3, 6, 7, and determines the value in the second position; and 
the third parity check, positions 4, 5, 6, 7, and determines the value in posi- 
tion four. This leaves positions 3, 5, 6, 7 as information positions. The results 
of writing down all possible binary numbers using positions 3, 5, 6, 7, and 
then calculating the values in the check positions 1, 2, 4, are shown 
in Table III. 

Thus a seven-position single error correcting code admits of 16 code sym- 
bols. There are, of course, 2 7 — 16 = 112 meaningless symbols. In some ap- 
plications it may be desirable to drop the first symbol from. the code to 
avoid the all zero combination as either a code symbol or a code symbol plus 
a single error, since this might be confused with no message. This would still 
leave 15 useful code symbols. 



W 



P&& 



gtffltfft 



■*4 



ft -ft 



ERROR DETECTING AND CORRECTING CODES 



153 



Tabus: III 



Position 


Decimal Value of, 
Symbol 


1 


2 


i 


4 


5 


6 


7 


























1 


1 





1 








1 


i 





1 





1 





1 





2 


1 














1 


1 


3 


1 








1 


1 








4 





1 








1 





1 


5 


1 


1 








1 


1 





6 











1 


1 


1 


1 


7 


1 


1 


1 














8 





. 


1 


1 








1 


9 


1 





1 


1 





1 





10 





1 


1 








1 


1 


H 





1 


1 


1 


1 





o \ 


12 


1 





1 





1 





1 } 


i u 








1 





1 


1 





\ 14 


1 


1 


1 


1 


1 


1 


1 


V 15 



_As an illustration of how this code " works" let us 
11110 corresponding to the decimal value 12 and dh 
the fifth position to a 0. We now examine the new symbol 

1110 00 



the symbol 
age the I in 



by the methods of this section to see how the error is located. From Table II 
the first parity check is over positions 1, 3 t 5, 7 and predicts a 1 for the first 
position while we find a there ; hence we write a 

: ' 1 . 

The second parity check is over positions 2, 3, 6, 7, and predicts the second 
puutkm correctly; hence we write a to the left of the 1, obtaining 

01. 

The third parity check is over positions 4, 5, 6, 7 and predicts wrongly; hence 
we write a 1 to the left of the 1, obtaining m 

10 1. \ 

This sequence of 0's and 1's regarded as a binary number is the number 5; 
hence the error is in the fifth position. The correct symbol is therefore ob- 
tained by changing the in the fifth position to a 1. 

4* Single Error CoitRECttNG Plus Double Error D^TECirofc Codes 

To construct a single error correcting phis double error detecting code we 
begin with a single error correcting code. To this code we add one more posi- 
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lion for checking aB the previous portion*, using an even parity ilmt T» 
see the operation of this code we hive to examine a number of cases: 

1, No errors. All parity checks, including the last, see satisfied. 

2, Single error. The last parity check fails in all suck sitaatione whether 
the error be in the information, the original check positions, or the hit 
check position. The original checking number fives the position of the 
error, when now the aero vakic means the last died ptrt^pn 

3, Two errors. In all each situations the last parity check is nt i,rf H, and 
the checking number indicates tone kind of error. 

As in illustration let us construct an eight -position code frasn the ] 
seven-position code. To do this we add an eighth p osition which is 
so that there are an even number of 1 'i in the eight posit lens Tims we add 
an eighth column to Table ID which has: 

TV 
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GENERAL THEORY 
5. A GaoiirrucAL Uqkl 



When examining various pro tafcas i connected with error detecting and 
correcting codes it is often convenient to Introduce a fmoetric model. 
The model used ben consists in fckndfying the various sequences of O's and 
l*s which are the symbols of a code with vertices of a unit a-dhnensionaJ 
cube. The code points, labelled x t y, s\ **• , form a subset of the set of a9 
vertices of the cube. 

Into this space of l m points we introduce a Os to m y or, as it is u easily 
called, a ss w Wr t D(s t y). The SsTswsttmw of the metric is based on the observa- 
tion that a single erne m a code pewit changes one oojordmate, two errors, 
*wo ooorvmaeBS} easa sa geneni a errata prnmsce a Qnrsrence sa a 
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Thus we define the distance D(x, y) between two points x and y as the num- 
ber of coordinates for which x and /are different. This is the same as the 
least number of edges which must be traversed in going from x to y. This 
distance function satisfies the usual three conditions for a metric, namely, 

D(x } y) = if and only if x = y 

D( Xj y) = D(y f x) > if x 9* y 

D(z, y) + D(y, z) > D(x, z) (triangle inequality). 

As an example we note that each of the following code points in the three- 
dimensional cube is two units away froj^n the others, 

01 
1 

111. 

To continue the geometric la^uage 3 a sphere of radius r about a point x 
is defined as all points which are at a distance r from the point x. Thus, in 
the above example, the first three code points are on a sphere of radius 2 
about the point (1, 1, 1). In fact, in this example any one code point may be 
chosen as the center and the other three will lie on the surface of a sphere 
of radius 2. / 

If all the code points are at a distance of at least 2 from each other, then it 
follows that any single error will carry a code point over to a point that is 
not a code point, and hence is a meaningless symbol. This in turn means that 
any single error is detectable. If the minimum distance between code points 
is at least three units then any single error will leave the point nearer to the 
correct code point than to any otter code point, and this means that any 
-mtfle error will be- correctable. This type of information is summarized in 
the following table: 

Table V 



Minimum 
Distance 



Meaning 



uniqueness 

single error detection 

single error correction 

single error correction plus double error detection 

double error correction 

Etc. 



(inversely, it is evident that, if we are to effect the detection and correc- 
tion listed, then all the distances between code points must equal or exceed 
the nnnimum distance listed. Thus the problem of finding suitable codes is 
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the same as that of finding subsets of points in the space which maintain at 
least the minimum distance condition. The special codes in sections 2, 3, 
and 4 were merely descriptions of how to choose a particular subset of points 
for minimum distances 2, 3, and 4 respectively. 

It should perhaps be noted that, at a given minimum distance, some of 
the correctability may be exchanged for more detectability. For example, a 
subset with minimum distance 5 may be used for: i 

a. double error correction, (with, of course, double error detection). ' 

b. single error correction plus triple error detection. 

c. quadruple error detection. 

Returning for the moment to the particular codes constructed in Part I 
we note that any interchanges of positions in a code do not change the code 
in any essential way. Neither does interchanging the O's and Ts in any posi- 
tion, a process usually called complementing. This idea is made more precise 
in the following definition: 

Definition. Two codes are said to be equivalent to each other if, by a finite 
number of the following operations, one can be transformed into the other: 

1. The interchange of any two positions in the code symbols. 

2. The complementing of the values in any position in the code symbols. 
This is a formal equivalence relation (~) since A ~ A; A ~ B implies 
B **" A; and A ~ B, B ~ C implies A ~ C. Thus we can reduce the study 
of a class of codes to the study of typical members of each equivalence class. 

In terms of the geometric model, equivalence transformations amount to 
rotations and reflections of the unit cube. 

6. Single Error Detecting Codes 

The problem studied in this section is that of packing the maximum num- 
ber of points in a unit ^-dimensional cube such that no two points are closer 
than 2 units from each other. We shall show that, as in section 2, 2 H ~ l points 
can be so packed, and, further, that any such optimal packing is equivalent 
to that used in section 2. 

To prove these statements we first observe that the vertices of the n- 
dimensional cube are composed of those of two (n — l)-dimensional cubes. 
Let A be the maximum number of points packed in the original cube. Then 
one of the two (n — l)-dimensional cubes has at least A/2 points. This cube 
being again decomposed into two lower dimensional cubes, we find that one 
of them has at least A/2* points. Continuing in this way we come to a two- 
dimensional cube having A/2 n ~* points. We now observe that a square can 
have at most two points separated by at least two units; hence the original 
^-dimensional cube had at most 2* _1 points not less than two units apart. 
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To prove the equivalence of any two optimal packings we note that, if 
the packing is optimal, then each of the two sub-cubes has half the points. 
Calling this the first coordinate we see that half the points have a and half 
have a 1. Hie next subdivision will again divide these into two equal groups 
having O's and l's respectively. After (n — 1) such stages we have, upon re- 
ordering the assigned values if there be any, exactly the first n — 1 positions 
of the code devised in section 2. To each sequence of the first * — 1 coordi- 
nates there exist * — 1 other sequences which differ from it by one co- 
ordinate. Once we fix the #*-th coordinate of some one point, say the origin 
which has all O's, then to maintain the known minimum distance of two 
units between code points the »~th coordinate is uniquely determined for all 
other code points. Thus the last coordinate is determined within a comple- 
mentation so that any optimal code is equivalent to that given in section 2. 

It is interesting to note that in these two proofs we have used only the 
assumption that the code symbols are all of length *». 

7, Single Error Correcting Codes 

It h*s probably been noted by the reader that, in the particular codes of 
Part I, a distinction was made between information and che\k positions, 

various 
ter we re- 
and 
>ntained 



fcthe 



while, in the geometric model, there is no real distinction betw< 
coordinates. To bring the two treatments more in line with each 
define a systematic code as a code whose symbol lengths are all eqi 

1. The positions checked are independent of the information 
in the symbol* k 

2. Hie checks are independent of each other. 

3. We use parity checks. 

This is equivalent to the earlier definition. To show this we form a matrix 
whose i-th row has l's in the positions of the i-th parity check and O's else- 
where. By assumption 1 the matrix is fixed and does not change from code 
symbol to code symbol. From 2 the rank of the matrix is k. This in turn 
means that the system can be solved for * of the positions expressed in 
terms of the other n ~ A positions. Assumption 3 indicates that in this 
solving we use the arithmetic in which 1+1*0, 

There exist non-systematic codes, but so far none have been found which 
for a given * and minimum distance d have more code symbols than a sys- 
tematic code. Section 9 gives ata example of a non-systematic code. * 

Turning to the main problem of this section we find from Table V that a 
single error correcting code has code points at least three units from each 
other. Thus each point may be surrounded by a sphere of radius 1 with no 
two spheres having a point in common. Each sphere has a center point and 
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n points on its surface, a total of n + 1 points. Thus the space of 2 n points 
can have at most: 

2 n 



n+ 1 



spheres. This is exactly the bound we found before in section 3. 

While we have shown that the special single error correcting code con- 
structed in section 3 is of minimum redundancy, we cannot show that all 
optimal codes are equivalent, since the following trivial example shows that 
this is not so. For n = 4 we find from Table I that m = 1 and k = 3. Thus 
there are »at most two code symbols in a four-position code. The following 
two optimal codes are clearly not equivalent: 

0000 , 0000 

mi and 111. 

8. Single Error Correcting Plus Double Error Detecting Codes 

In this section we shall prove that the codes constructed in section 4 are 
of minimum redundancy. We have already shown in section 4 how, for a 
minimum redundancy code of n — 1 dimensions with a minimum distance 
of 3, we can construct an n dimensional code having the same number of 
code symbols but with a minimum distance of 4. If this were not of minimum 
redundancy there would exist a code having more code symbols but with 
the same n and the same minimum distance 4 between them. Taking this 
code we remove the last coordinate. This reduces trie dimension from n to 
n — 1 and the minimum distance between code symbols by, at most, one 
unit, while leading the number of code symbols the same. This contradicts 
the assumption that the code we began our construction with was of mini- 
mum reduncancy. Thus the codes of section 4 are of minimum redundancy. 

This Is a special case of the following general theorem: To any minimum 
redundancy code of N points in n — 1 dimensions and having a minimum 
distance oi 2k — 1 there corresponds a minimum redundancy code of X 
points in n dimensions having a minimum distance of 2k, and conversely. 
To construct the n dimensional code from the n — 1 dimensional code we 
simply add a single n-th coordinate which is fixed by an even parity check 
over the n positions. This also increases the minimum distance by*l for 
the following reason: Any two points which, in the w — 1 dimensional code, 
were at a distance 2k — 1 frdm each other had an odd number of differences 
between their coordinates. Thus the parity check was set oppositely for the 
two points, increasing the distance between them to 2k, The additional co- 
ordinate could not decrease any distances, so that all points in the code are 
now at a minimum distance of 2k. To go in the reverse direction we simply 
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drop one coordinate from the » dimensional code. This reduces the minimum 
distance of 2k to 2k — 1 while leaving A r the same. It is clear that if one 
code is of minimum redundancy then the other is, too. 

9. Miscellaneous Observations 

For the next case, minimum distance of five units, one can surround each 
code point by a sphere of radius 2. Each sphere will contain 

1 + C(*, 1) + C(*, 2) 

points, where C(n, k) is the binomial coefficient, so that an upper bound on 
the number of code points in a systematic code is 

1 + C(n, 1) + C(«, 2) " »« + n + 2 ~ 2 ' 

This bound is too high. For example, in the case of n «■ 7, we find that 
w = 2 so that there should be a code with four code points. The maximum 
possible, as can be easily found by trial and error, is two. 

In a similar fashion a bound on the number of code points may be found 
whenever the minimum distance between code points is an odd number. 
A bound on the even cases can then be found by use of the general theorem 
of the preceding section. These bounds are, in general, too high, as the above 
example shows. 

If we write the bound on the number of code points in a unit cube of dimen- 
sion n and with minimum distance d between them as B(n, d), then the 
information of this type in the present paper may be summarized as follows: 

B{n % 1) * 2* 

B(n, 2) « 2*- 1 

2 * 



B(n t 3) - 2 m < 



* + 1 



B(n, 4) = 2" < — 
n 

B(n - 1, 2k - 1) - B(n, 2k) 
B(n, 2* - 1) - 2" < 



x 1 + C(n, 1) + ■ • • + C(n, k - 1) ' 

* 

While these bounds have been attained for certain cases, no general 
methods have yet been found for contructing optimal codes when the mini- 
mum distance between code points exceeds four units, nor is it known 
whether the bound is or is not attainable by systematic codes. 
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We have dealt mainly with systematic codes. The existence of non-sys- 
tematic codes is proved by the following example of a single error correcting 
code with n =6. 



f 



00 00 
10 10 1 
10 110 
1110 
10 11 
111111. 

Hie all symbol indicates that any parity check must be an even one. 
The all 1 symbol indicates that each parity check must involve an even num- 
ber of positions. A direct comparison indicates that since no two columns 
are the same the even parity checks must involve four or six positions. An 
examination of the second symbol, which has three Ts in it, indicates that 
no six-position parity check can exist. Trying now the four-position parity 
checks we find that 

12 5 6 | 

2 3 4 5 

are two independent parity checks and that no third one is independent of 
these two. Two parity checks can at most locate four positions, and, since 
there are six positions in the code, these two parity checks are not enough 
to locate any single error. The code is, however, single error correcting since 
it satisfies the minimum distance condition of three units. 

The only previous work in the field of error correction that has appeared 
in print, so far as the author is aware, is that of M. J. E. Golay. 4 

4 M. J. £. Golay, Correspondence, Notes on Digital Coding, Proceedings of the I.R.E., 
Vol. 37, p. 657, June 1949. 
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